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ABSTRACT 

A  number  of  technologies  are  evolving  that  will 
help  formulate  more  adaptive  and  robust  network 
architectures  intended  to  operate  in  dynamic, 
mobile  environments.  One  technology  area, 
mobile  ad  hoc  networking  (MANET)  enables  self¬ 
organizing,  multi-hop  heterogeneous  network 
routing  services  and  organization.  Such 
technology  is  important  in  future  DoD  networking, 
especially  in  the  forward  edge  of  the  battlespace 
where  self-organizing,  robust  networking  is 
needed.  A  second  technology  area.  Multi-Agent 
Systems  (MAS)  can  enable  autonomous,  team- 
based  problem  solving  under  varying 
environmental  conditions.  Previous  work  done  in 
MAS  has  assumed  relatively  benign  wired  network 
behavior  and  inter-agent  communications 
characteristics  that  may  not  be  well  supported  in 
MANET  environments.  In  addition,  the  resource 
costs  associated  with  performing  inter-agent 
communications  have  a  more  profound  impact  in  a 
mobile  wireless  environment.  The  combined 
operation  of  these  technology!  areas,  including 
cross-layer  design  considerations,  has  largely 
been  unexplored  to  date.  This  paper  describes 
ongoing  research  to  improve  the  ability  of  these 
technologies  to  work  in  concert.  An  outline  of 
various  design  and  system  architecture  issues  is 
first  presented.  We  then  describe  models,  agent 
systems.  MANET  protocols,  and  additional 
components  that  are  being  applied  in  our 
research.  We  present  an  analysis  method  to 
measure  agent  effectiveness  and  early  evaluations 
of  working  prototypes  within  MANET 
environments.  We  conclude  by  outlining  some 
open  issues  and  areas  of  further  work. 

BACKGROUND  AND  MOTIVATION 

Mobile  Ad  hoc  Networking  (MANET)  technology 
is  designed  to  provide  effective  dynamic  Internet 
Protocol  (IP)  routing  in  network  environments 


where  "change  is  the  norm".  Change  may  be 
topological  due  to  mobility  or  behavioral  due  to 
other  wireless  environmental  effects  (e.g.,  energy 
conservation,  channel  fading,  etc)  [CM99],[P01]. 
In  addition,  multi-agent  system  (MAS)  technology 
is  well  suited  for  dynamic,  distributed  problem 
solving  in  which  distributed  software  agents  are 
able  to  both  sense  and  act  within  complex 
environments  [W02].  MAS-based  architectures 
may  be  very  valuable  in  the  fliture  forward 
battlespace  to  provide  distributed  teamwork-based 
solutions  to  complex  problems.  However,  at 
present,  MAS  design  and  performance  tradeoffs  in 
disruptive  and  potentially  mobile  networks  are 
largely  unexplored. 

PROBLEM  SPACE 

MANET  is  planned  for  fliture  deployment  in  the 
forward  edge  of  the  battlespace.  There  has  been 
some  significant  advancement  in  the  development 
of  MANET  solutions  in  recent  years,  but  much 
work  remains  to  be  done.  Overall,  there  remains 
limited  experience  in  using/adapting  upper  layer 
protocols  and  applications  in  these  environments 
[B04].  Also  more  work  is  needed  in  developing 
and  adapting  multicast  routing  in  these 
environments  to  better  support  group-oriented 
communications.  Intelligent  multi -agent  systems 
are  also  of  interest  in  fliture  DoD  systems.  These 
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Figure  1:  Dynamic  MAS  Benefits 
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systems  are  composed  of  autonomous  or 
cooperating  software  entities;  in  the  latter  case 
these  entities  may  be  goal-driven  and  able  to 
coordinate  (communicate)  as  part  of  a  team  to 
reach  tire  goals. 

Figure  1  shows  the  potential  benefit  of  MAS  for 
more  autonomous  and  flexible  problem  solving  in 
complex  environments.  Some  possible  examples 
of  future  MAS  applications  include  distributed 
command  and  control  support  software,  network 
management,  sensor  networking,  and  mobile 
cooperative  robotics.  When  considering  MANET 
deployment  in  highly  stressed  networks,  the  design 
of  distributed  and  cooperative  agent-based  systems 
is  of  interest.  This  is  primarily  due  to  the  flexible 
problem  solving  approach  offered  by  MAS,  which 
provides  a  robust  solution  to  address  the 
challenges  faced  in  MANET  environments. 
Previous  design  work  has  assumed  a  benign  wired 
network  behavior;  not  MANET  environments.  In 
MANET  environments  peer  agents  experience 
increased  topological  dynamics,  intennittent 
connectivity,  and  reduced  reliability.  The  cost  of 
interagent  communications  is  also  high;  therefore 
communication  resources  in  a  wireless  MANET 
should  be  utilized  in  the  most  effective  means 
possible.  For  example,  multi-party  coordination 
and  communication  frameworks  should  be  well 
supported. 

Separately  MAS  and  MANET  encompass  two 
challenging  research  areas.  Our  focus  is  to  begin 
improving  the  performance  capability  of  MAS 
applications  running  in  MANET  network 
environments.  This  includes  tire  following 
subgoals: 

•  improve  crosslayer  design  performance 
between  MANET  and  MAS  layers. 

•  investigate  MAS  design  robustness  in 
stressed  network  environments. 

•  develop  and  test  new  MANET  network 
services  and  modeling  where  needed  to 
support  anticipated  MAS  operation. 

ISSUES  AND  PROJECT  APPROACH 

Figure  2  represents  an  example  of  the  architectural 
layering  concept  that  we  are  targeting  in  this 
research  effort.  The  goals  are  to  identify  and  fill 
technology  and  protocol  gaps  that  may  exist  in 
Figure  2  and  improve  any  flawed  design 
interactions  between  the  layers.  At  the  same  time. 


we  wish  to  maintain  layered  software  and  network 
protocol  abstractions  wherever  possible. 


Figure  2:  Layered  Design  Issues 

At  tlie  MAS  layer,  we  are  interested  in  better 
multiagent  designs  to  improve  the  potential  for 
operation  in  MANET  environments.  In  doing  so, 
we  are  investigating  tlie  network  service 
assumptions  and  robustness  requirements  at  the 
agent  layer.  Special  attention  is  being  paid  to 
interagent  communication  design  including 
efficiency  and  flexibility  of  agent  teamwork.  An 
important  issue  is  establishing  measures  of 
effectiveness  applied  to  agent  designs  within  these 
environments.  At  present,  we  are  studying 
teamwork  problem  solving  effectiveness  vs. 
perfonnance  metrics  such  as  increasing  network 
size,  mobility,  and  decreasing  reliability. 

Figure  2  also  shows  middleware  as  a  potential 
design  layer  between  MAS  and  MANET. 
Middleware  plays  a  role  as  an  important  network 
design  abstraction  often  used  by  network 
application  designers,  including  designers  of 
network  agent  software.  Dynamic  service 
discovery  and  other  higher  layer  services  are  also 
often  provided  by  a  middleware  layer.  Examples 
of  middleware  systems  include  JXTA,  Gnutella, 
and  Peer-to-Peer  Simplified  (P2PS)  [W05]. 
Despite  their  attractiveness  as  design  abstractions, 
middleware  frameworks  are  not  an  engineering 
panacea,  especially  in  wireless  environments.  For 
example,  middleware  can  take  advantage  of 
efficient  network  communication  abstractions  such 
as  multicast  but  the  present  designs  often  assume 
static  topology  relationships  or  neighborhood 
subnet  assumptions  more  consistent  with  wired 
network  deployments.  There  are  also  robustness 
and  efficiency  tradeoffs  between  classes  of 
middleware  services  that  are  unique  to  MANET 


2 


environment  scenarios.  Middleware  or  peer-to- 
peer  layers  also  often  provide  some  autonomous 
organization  and  identification  independent  of  the 
underlying  network  addresses  such  as  the  Internet 
Protocol  (IP)  stack  provides.  We  are  interested  in 
studying  the  potential  functional  redundancy  and 
inefficiencies  of  such  layered  abstractions  when 
operating  in  MANET  environments.  We  envision 
an  enhanced  approach  in  which  more  efficient 
lower  layer  MANET  network  services  (e.g., 
multicast)  may  be  utilized  by  middleware  when 
operating  in  stressed  wireless  ad  hoc  scenarios. 
This  will  maintain  a  degree  of  abstraction  desired 
of  middleware  but  will  provide  more  efficient  and 
effective  cross-layer  components  for  MANET  use. 
Figure  2  also  shows  the  MANET  layer  as  the 
bottom  layer  of  the  architecture  providing  ad  hoc 
multiple-hop  routing  within  a  wireless  network. 
The  figure  also  depicts  the  fact  that  tire  scenario 
may  be  very  heterogeneous  between  layers.  For 
instance,  some  nodes  may  only  provide  network 
and  possibly  middleware  services  for  other  nodes 
that  are  operating  as  agent  nodes.  In  other  cases,  a 
physical  node  may  operate  as  an  agent, 
middleware,  and  MANET  node  simultaneously. 

As  envisioned  the  MANET  protocol  layer  is 
providing  the  following  two  main  functions: 

•  Dynamic  IP  network  routing 

•  Some  network  autoconfiguration  support 

The  MANET  dynamic  IP  routing  is  performing 
both  unicast  and  multicast  forwarding  of  data 
packets.  We  are  presently  looking  at  the 

application  of  MANET  proactive  unicast  protocols 
in  this  environment.  We  expect  MANET 
proactive  to  provide  lower  average  delay  between 
nodes  and  more  predictable  network  overhead 
under  stressed  conditions.  The  dense  many-to- 
many  traffic  patterns  involved  in  interagent 
teamwork  communications  likely  favor  a  more 
proactive  approach  but  scaling  a  proactive 
approach  can  be  challenging  and  future  work  in 
that  area  is  needed.  There  is  also  nothing  in  our 
architecture  concept  to  preclude  the  substitution  of 
reactive  protocols  as  well. 

Because  of  the  MAS  teamwork  model  involving 
group  communications,  multicast  MANET  routing 
is  also  of  particular  interest.  We  are  adapting 
ongoing  work  on  Simplified  Multicast  Forwarding 
(SMF)  protocol  [MDC04]  to  our  evaluation 


models.  An  effective  MANET  multicast  routing 
capability  will  be  important  since  interagent  group 
communications  and  persistent  sharing  of  team 
environment  or  plan  infonnation  is  called  for  by 
many  interagent  designs.  Multicast  may  also  be 
adjusted  and  optimized  for  more  localized  agent 
communication  designs  by  changing  scope  range 
of  the  multicast  dissemination. 


MODELING  APPROACH  AND  TOOLS 


Due  to  analytical  complexity,  research  of  MANET 
systems  is  often  done  by  simulation  and  sometimes 
via  emulation.  Researchers  model  the  related 
dynamic  network  environments,  network  protocol 
stacks,  applications,  and  node  mobility  witliin 
specialized  simulation  or  emulation  systems.  In 
our  work,  we  have  extended  some  existing 
MANET  test  environments  to  include  the 
introduction  of  non-abstracted  agent  software  and 
middleware  components. 
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Figure  3;  Example  Modeling  Approach 

Figure  3  depicts  part  of  the  modeling  approach  and 
toolset  we  have  developed.  As  shown,  we  use  a 
cross  platform  prototype  code  development 
approach  that  results  in  the  same  software  able  to 
run  in  both  simulation  and  emulation  environments 
[CMW03].  The  composite  system  under 
examination  includes  the  agent  software  at  the  top 
layer,  optional  middleware,  and  then  associated 
network  stack  and  MANET  protocols  below.  At 
the  MANET  layer,  we  have  added  the  addition  of 
SMF  functionality  providing  an  optional  multicast 
routing  capability  within  the  MANET 
environment.  Agents  are  useful  in  real  world 
environments  because  they  both  sense  and  react  to 
complex  environments.  To  better  support 
environmental  modeling  we  have  adapted  an 
existing  set  of  MANET  tools  to  provide  dynamic 
environment  control  and  stimulus  to  the  MAS 
simulations  [D04].  In  the  past  the  environment 
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channel  was  normally  used  to  control  MANET 
node  movement  and  position  information  in 
mobile  simulations,  but  it  is  now  also  a  channel 
that  can  provide  local  environmental  information 
used  by  the  agents  (e.g.,  localized  sensor  stimulus, 
human  input,  etc).  Once  again  the  set  of  scenarios, 
traffic  tools,  and  visualization  tools  equally  apply 
to  both  simulation  and  emulation  testing.  In 
Figure  3,  the  optional  middleware  modeling  layer 
is  represented  by  the  P2PS  protocol  layer  [T05]. 
We  have  modified  P2PS  interface  code  to  run  in  a 
portable  way  in  the  real  world  and  within  the  ns2 
simulation  environment.  Agents  may  use  the 
middleware  services  or  can  optionally  directly 
communicate  through  standard  network  and 
MANET-optimized  network  layers.  We  are  also 
working  on  adding  a  multicast  pipe  capability  to 
P2PS  to  be  able  to  use  the  SMF  MANET  routing 
and  possibly  future  additional  reliable  multicast 
transport  services. 

EARLY  AD  HOC  MAS  STUDY  RESULTS 

As  we  have  discussed  there  are  numerous  design 
frameworks  and  approaches  being  developed  for 
MAS  and  the  application  areas  are  varied. 
Because  of  the  rich  taxonomy  of  application  areas 
for  MAS  this  a  ‘'no  one  size  fits  all”  design  space. 
We  have  chosen  to  scope  our  investigation  down 
to  an  area  of  team-based  problem  solving,  with 
initial  emphasis  on  the  study  of  distributed  role 
allocation  algorithms  and  their  effectiveness 
towards  achieving  team  goals.  We  accomplished 
early  work  in  adapting  existing  agent-basesd 
systems  and  frameworks  including  Machinett 
[MA],  Control  of  Agent-Based  System  (CoABS) 
grid,  and  REPAST  to  enable  early  analysis  and 
refinement  of  the  algorithms  prior  to  integration 
with  MANET  environments  [AM03]. 

An  example  of  our  team-based  problem  solving 
scenario  is  the  prey/predator  pursuit  problem.  The 
prey/predator  problem  is  a  canonical  example  of 
agent  teamwork  in  the  literature  and  we  are 
adapting  this  model  to  further  study  by  including  a 
MANET  communications  model.  We  have 
measured  coordination  (and  hence  teamwork) 
quality  as  a  function  of  interactions  between  agents 
(messages  and  collisions)  to  accomplish  the  task  of 
capturing  the  preys. 

Figure  4  is  an  example  visualization  snapshot  of  an 
emulation  scenario.  Within  figure  4,  the  icons  of 
the  soldiers  represent  predator  and  the  monsters 


represent  prey.  The  circle  depicted  around  the 
prey  represents  the  capture  zone  of  the  preys.  In 
this  example,  it  takes  4  predators  to  successfully 
capture  a  prey  within  the  capture  zone.  Tlierefore 
agents  are  required  to  perform  teamwork  in  order 
to  satisfy  tlie  requirement  of  four  complementary 
and  unique  roles  beuig  taken  to  capture  the  prey. 
The  links  between  the  predators  represents  the 
real-time  network  topology  being  calculated  by  the 
MANET  routing  protocol  (in  this  case  a  variant  of 
OLSR).  The  predators  and  preys  are  mobile 
witliin  the  scenario  and  the  predators  use  the 
MANET  to  form  a  mobile  network  to  allow  for 
interagent  coordination  and  communication.  The 
agents  support  the  dynamic  allocation  of  roles  and 
agent  teamwork  by  communicating  environmental 
state  and  or  local  intention  across  the  MANET 
network. 


Figure  4:  Emulation  Visualization  Snapshot 

In  early  studies,  we  began  by  examining  various 
role  allocation  algorithms  in  a  more  limited 
simulation  environment  (i.e.,  REPAST).  We 
induced  simulated  message  loss  and  limited 
communication  range  (to  crudely  represent  facets 
of  MANET)  to  examine  performance  of  the 
various  role  allocation  approaches  [AMC05].  The 
initial  results  from  those  experiments  showed  that 
distributed  coordination  by  sharing  state  among 
agents  significantly  outperformed  algorithms 
based  solely  on  communicating  role  switching 
with  more  localized  and  statistical  methods.  The 
interesting  fact  was  that  this  advantage  remained 
true  with  high  network  loss  rates  as  well  as  when 
the  system  scaled  in  terms  of  numbers  of  agents. 
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Figure  5 :  Role  Allocation  Simulations 

We  are  now  pursuing  similar  experiments  within  a 
more  detailed  emulation  and  simulation 
environment.  Tire  mobile  topology,  routing 
performance,  contention,  and  congestion  are  being 
modeling  with  working  protocol  code  and  the 
wireless  network  environment  and  protocol  stack 
will  be  more  realistic.  Figure  5  shows  an  early  set 
of  results  demonstrating  that  the  Distributed 
Constraint  Optimization  (DCO)  based  upon  the 
Flungarian  algorithm  continues  to  outperform  the 
more  localized  statistical  role  allocation  methods’ 
within  the  more  detailed  emulation  environment. 


Figure  6:  Early  Simulation  and  Emulation  Results 

MANET  MULTICAST  PROGRESS  AND 
AGENT  USE 

We  have  made  significant  recent  progress  in 
flirther  developed  and  applying  SMF  for  multicast 
routing  within  this  project  [MDC04].  New  and 
improved  relay  set  algorithms  have  been 
introduced  into  SMF  and  a  simulation  model  has 
been  completed  and  tested  with  larger  networks 
than  supported  within  the  emulator.  Some  agent 
role  allocation  approaches  and  designs,  such  as  the 


’  Distributed  Stochastic  Algorithm  (DSA),  Simple 
Distributed  Improvement  (SDI) 


Hungarian  algorithm  variant  require  significant 
interagent  sharing  of  local  environment 
infonnation.  These  types  of  algorithms  often  find 
more  optimal  solutions  to  a  complex  problem 
because  of  the  more  accurate  agent  view. 
However,  this  is  accomplished  at  the  cost  of 
increased  inter-agent  cormnunications.  SMF 
reduces  network  overhead  by  providing  an 
efficient  means  of  multicast  forwarding  within  a 
dynamic,  wireless  network  area  making  such 
solutions  more  viable  within  MANETs.  We  are 
presently  exploring  the  tradeoffs  between 
improved  agent  role  allocation  performance  and 
the  cost  of  this  additional  overhead  in  various 
scenarios.  The  example  of  Figure  6  uses  SMF 
within  the  emulation  experiment  and  shows  that 
the  additional  network  overhead  is  being  utilized 
in  an  effective  way.  As  mentioned,  protocols  like 
SMF  can  decrease  the  burden  of  overhead  within 
the  MANET  operating  area  and  make  more 
effective  agent  communication  possible.  We  plan 
to  study  this  tradeoff  more  as  the  network  and  the 
MAS  scale  in  size. 


SMr  vs.  PSPS  da'ta  rai:#  oemiear  1  s len 


Figure  7;  P2PS  Middleware  (unicast)  vs.  SMF 
pipes 

Figure  7  shows  some  early  analysis  of  network 
overhead  with  seven  agents,  two  preys,  and  one 
scout.  All  nodes  are  mobile  and  the  agents  and 
scout  use  MANET  protocols  to  dynamically  route 
data  within  the  changing  topology.  In  one  case, 
we  use  P2PS  as  a  peer-to-peer  protocol  layer 
between  agents.  P2PS  provides  both  unicast 
messaging  pipes  and  service  discovery  for  the 
MAS  software.  In  the  second  case,  we  applied 
SMF  as  a  mobile  multicast  forwarding  directly 
between  agents.  This  is  more  efficient  as  a  team 
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communication  method  and  the  curve  represents 
the  resulting  savings  in  network  overhead  from 
using  SMF  vs.  unicast  P2PS.  In  future  work,  we 
hope  to  combine  the  use  of  SMF  and  P2PS  so  that 
the  middleware  can  take  advantage  of  efficient 
multicast  forwarding  when  available  in  MANETs. 

FUTURE  WORK 

Part  of  the  challenge  of  studying  MAS  systems  is 
to  measure  the  effectiveness  of  performance.  This 
is  even  more  challenging  within  highly  complex 
environments  with  competing  goals.  Using  the 
outlined  simulation  and  emulation  test 
environments,  future  work  will  develop  a  set  of 
more  comprehensive  tests  and  measures  for  a 
variety  of  agent  teamwork  scenarios  beyond  the 
present  role  allocation  problems.  Also,  MANET 
environment  variables  can  vary  greatly  and 
significantly  affect  the  outcome  of  evaluation 
results.  As  a  network  environment  degrades,  there 
may  be  no  one  best  answer,  so  in  future  work,  we 
plan  to  investigate  the  possibility  of  agent 
adaptation  to  network  conditions.  As  the  network 
degrades  in  terms  of  delay,  capacity,  or 
connectivity  the  agent  teams  may  adapt  their 
distributed  decision  methods  to  better  match 
network  conditions. 

CONCLUSION 

In  summery,  we  have  presented  some  recent  work 
being  performed  in  the  area  of  MAS  and  MANET 
technology  research.  The  main  scope  of  out  work 
is  studying  team-based  MAS  problem  solving 
using  dynamic  role  allocation  within  the  context  of 
MANET  environments.  Early  analysis  has  shown 
that  MAS  teamwork  communications  within 
MANET  can  be  improved  through  better  design 
between  the  layers.  The  use  and  adaptation  of 
MANET  multicast  is  one  example.  We  have  also 
discovered  design  problem  with  present 
middleware  systems  (e.g.,  JXTA[JX])  often  used 
by  agent  to  provide  abstracted  service  discovery 
and  communication  services.  Further 
improvements  are  being  examined  relating  to 
MANET  application  scenarios. 

We  have  developed  and  described  a  set  of  test 
tools  including  both  simulation  and  emulation  to 
further  study  detailed  scenarios  of  combined  MAS 
and  MANET  operation.  An  environment  channel 
was  developed  to  better  control  and  exercise 
distributed  MANET  experiments  and  to  provide 


agents  with  simulated  external  stimulus  outside  the 
network  domain.  We  have  adapted  a  lightweight 
middleware  framework  for  study  and  we  are 
examining  services  discovery  and  other  features 
for  possible  interagent  use  in  MANET.  We  have 
also  adopted  the  SMF  multicast  routing  prototype 
for  MANET  and  we  are  applying  this  as  a  means 
for  more  efficient  inter-agent  group  multicasting. 
Presently,  we  are  beginning  detailed  emulation  and 
simulation  studies  involving  prey/predator  agent 
scenarios  within  both  the  MANET  simulation  and 
emulation  environments.  We  plan  to  study  the 
tradeoffs  of  various  distributed  role  allocation 
approaches  and  the  effectiveness  of  various 
MANET  protocol  enhancements  on  agent 
performance. 

Early  results  confirm  our  initial  hypothesis  that 
classic  MAS  and  middleware  communication 
design  assumptions  may  be  invalid  when  applied 
in  MANET  environments.  More  recently,  other 
researchers  are  examining  related  areas  and  have 
reported  similar  issues,  and  Additional  research 
will  be  aimed  towards  better  understanding  the 
performance  tradeoffs  and  improving  designs  in  a 
variety  of  MAS  scenarios  and  MANET 
environments. 
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